Speech production and the management/prediction of hearing loss

ABSTRACT

A method, including obtaining hearing data and speech data for a statistically significant number of individuals, and analyzing the obtained hearing data and speech data using a neural network to develop a predictive algorithm for hearing loss based on the results of the analysis, wherein the predictive algorithm predicts hearing loss based on input indicative of speech of a hearing impaired person who is not one of the individuals.

BACKGROUND

Hearing loss, which may be due to many different causes, is generally oftwo types: conductive and sensorineural. Sensorineural hearing loss isdue to the absence or destruction of the hair cells in the cochlea thattransduce sound signals into nerve impulses. Various hearing prosthesesare commercially available to provide individuals suffering fromsensorineural hearing loss with the ability to perceive sound. Oneexample of a hearing prosthesis is a cochlear implant.

Conductive hearing loss occurs when the normal mechanical pathways thatprovide sound to hair cells in the cochlea are impeded, for example, bydamage to the ossicular chain or the ear canal. Individuals sufferingfrom conductive hearing loss may retain some form of residual hearingbecause the hair cells in the cochlea may remain undamaged.

Individuals suffering from hearing loss typically receive an acoustichearing aid. Conventional hearing aids rely on principles of airconduction to transmit acoustic signals to the cochlea. In particular, ahearing aid typically uses an arrangement positioned in the recipient'sear canal or on the outer ear to amplify a sound received by the outerear of the recipient. This amplified sound reaches the cochlea causingmotion of the perilymph and stimulation of the auditory nerve. Cases ofconductive hearing loss typically are treated by means of boneconduction hearing aids. In contrast to conventional hearing aids, thesedevices use a mechanical actuator that is coupled to the skull bone toapply the amplified sound.

In contrast to hearing aids, which rely primarily on the principles ofair conduction, certain types of hearing prostheses commonly referred toas cochlear implants convert a received sound into electricalstimulation. The electrical stimulation is applied to the cochlea, whichresults in the perception of the received sound.

SUMMARY

In accordance with an exemplary embodiment, there is a method,comprising obtaining hearing data and speech data for a statisticallysignificant number of individuals, and analyzing the obtained hearingdata and speech data using machine learning to develop a predictivealgorithm for hearing loss based on the results of the analysis, whereinthe predictive algorithm predicts hearing loss based on input indicativeof speech of a hearing impaired person who is not one of theindividuals.

In accordance with another embodiment, there is a method, comprisingobtaining data based on speech of a person; and analyzing the obtaineddata based on speech using a code of and/or from a machine learningalgorithm to develop data regarding hearing loss of the person, whereinthe machine learning algorithm is a trained system trained based on astatistically significant population of hearing impaired persons

In accordance with another exemplary embodiment, there is a method,comprising obtaining data based on speech of a person, and developing aprescription and/or a fitting regime for a hearing prosthesis based onthe obtained data.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments are described below with reference to the attached drawings,in which:

FIG. 1A is a perspective view of an exemplary multimodal hearingprosthesis according to an exemplary embodiment;

FIG. 1B is another view of the exemplary multimodal hearing prosthesispresented in FIG. 1A;

FIG. 1C provides additional details of the exemplary multimodal hearingprosthesis of FIG. 1B;

FIG. 2 depicts an exemplary flowchart for an exemplary method accordingto an exemplary embodiment;

FIG. 3 depicts an exemplary conceptual schematic for a system accordingto an exemplary embodiment;

FIG. 4 depicts another exemplary flowchart for another exemplary methodaccording to another exemplary embodiment;

FIG. 5 depicts another exemplary conceptual schematic for another systemaccording to an exemplary embodiment;

FIG. 6 depicts by way of conceptual functional schematic the progressionof a machine learning algorithm as it is trained and an exemplaryresulting offspring thereof;

FIG. 7 depicts an exemplary schematic representing in graphical terms amethod utilizing a neural network and the results thereof according toan exemplary embodiment;

FIGS. 8-11 present exemplary flowchart for exemplary methods accordingto some exemplary embodiments;

FIG. 12 depicts an exemplary schematic conceptually illustrating asystem according to an exemplary embodiment;

FIG. 13 depicts a schematic functionally representing an exemplarysystem; and

FIG. 14 depicts a schematic functionally representing another exemplarysystem.

DETAILED DESCRIPTION

FIG. 1A is a perspective view of an exemplary multimodal prosthesis inwhich the present invention may be implemented. The ear 99 includesouter ear 201, middle ear 205, and inner ear 207 are described nextbelow, followed by a description of an implanted multimodal system 200.Multimodal system 200 provides multiple types of stimulation, i.e.,acoustic, electrical, and/or mechanical. These different stimulationmodes may be applied ipsilaterally or contralaterally. In the embodimentshown in FIG. 1A, multimodal implant 200 provides acoustic andelectrical stimulation, although other combinations of modes can beimplemented in some embodiments. By way of example and not by way oflimitation, a middle-ear implant can be utilized in combination with thecochlear implant, a bone conduction device can be utilized incombination with the cochlear implant, etc.

In a person with normal hearing or a recipient with residual hearing, anacoustic pressure or sound wave 203 is collected by outer ear 201 (thatis, the auricle) and channeled into and through ear canal 206. Disposedacross the distal end of ear canal 206 is a tympanic membrane 204 whichvibrates in response to acoustic wave 203. This vibration is coupled tothe oval window, fenestra ovalis 215 through three bones of middle ear205, collectively referred to as the ossicles 217 and comprising themalleus 213, the incus 209, and the stapes 211. Bones 213, 209, and 211of middle ear 205 serve to filter and transfer acoustic wave 203,causing oval window 215 to articulate, or vibrate. Such vibration setsup waves of fluid motion within cochlea 232. Such fluid motion, in turn,activates tiny hair cells (not shown) that line the inside of cochlea232. Activation of the hair cells causes appropriate nerve impulses tobe transferred through the spiral ganglion cells (not shown) andauditory nerve 238 to the brain (not shown), where such pulses areperceived as sound.

In individuals with a hearing deficiency who may have some residualhearing, an implant or hearing instrument may improve that individual'sability to perceive sound. Multimodal prosthesis 200 may comprisesexternal component assembly 242 which is directly or indirectly attachedto the body of the recipient, and an internal component assembly 244which is temporarily or permanently implanted in the recipient. Externalcomponent assembly is also shown in FIG. 1B. In embodiments of thepresent invention, components in the external assembly 242 may beincluded as part of the implanted assembly 244, and vice versa. Also,embodiments of the present invention may be used with implantedmultimodal system 200 which are fully implanted.

External assembly 242 typically comprises a sound transducer 220 fordetecting sound, and for generating an electrical audio signal,typically an analog audio signal. In this illustrative embodiment, soundtransducer 220 is a microphone. In alternative embodiments, soundtransducer 220 can be any device now or later developed that can detectsound and generate electrical signals representative of such sound.

External assembly 242 also comprises a signal processing unit, a powersource, and an external transmitter unit. External transmitter unit 206comprises an external coil 208 and, preferably, a magnet 206 secureddirectly or indirectly to the external coil 208. Signal processing unitprocesses the output of microphone 220 that is positioned, in thedepicted embodiment, by outer ear 201 of the recipient. Signalprocessing unit generates coded signals, referred to herein as astimulation data signals, which are provided to external transmitterunit 206 via a cable 247 and to the receiver in the ear 250 via cable252. FIG. 1C provides additional details of an exemplary receiver 250.The overall component containing the signal processing unit is, in thisillustration, constructed and arranged so that it can fit behind outerear 201 in a BTE (behind-the-ear) configuration, but may also be worn ondifferent parts of the recipient's body or clothing.

In some embodiments, the signal processor may produce electricalstimulations alone, without generation of any acoustic stimulationbeyond those that naturally enter the ear. While in still furtherembodiments, two signal processors may be used. One signal processor isused for generating electrical stimulations in conjunction with a secondspeech processor used for producing acoustic stimulations.

As shown in FIGS. 1B and 1C, a receiver in the ear 250 is connected tothe signal processor through cable 252. Receiver in the ear 250 includesa housing 256, which may be a molding shaped to the recipient. Insidereceiver in the ear 250 there is provided a capacitor 258, receiver 260and protector 262. Also, there may be a vent shaft 264 (in someembodiments, this vent shaft is not included). Receiver in the ear maybe an in-the-ear (ITE) or completely-in-canal (CIC) configuration.

Also, FIG. 1B shows a removable battery 270 directly attached to thebody/spine of the BTE device. As seen, the BTE device in someembodiments control buttons 274. In addition, the BTE may house a powersource, e.g., zinc-air batteries. The BTE device may have an indicatorlight 276 on the earhook to indicate operational status of signalprocessor. Examples of status indications include a flicker whenreceiving incoming sounds, low rate flashing when power source is low orhigh rate flashing for other problems.

Returning to FIG. 1A, internal components 244 comprise an internalreceiver unit 212, a stimulator unit 226 and an electrode assembly 218.Internal receiver unit 212 comprises an internal transcutaneous transfercoil (not shown), and preferably, a magnet (also not shown) fixedrelative to the internal coil. Internal receiver unit 212 and stimulatorunit 226 are hermetically sealed within a biocompatible housing. Theinternal coil receives power and data from external coil 208. Cable orlead of electrode assembly 218 extends from stimulator unit 226 tocochlea 232 and terminates in an array 234 of electrodes 236. Electricalsignals generated by stimulator unit 226 are applied by electrodes 236to cochlea 232, thereby stimulating the auditory nerve 238.

In one embodiment, external coil 208 transmits electrical signals to theinternal coil via a radio frequency (RF) link. The internal coil istypically a wire antenna coil comprised of at least one and preferablymultiple turns of electrically insulated single-strand or multi-strandplatinum or gold wire. The electrical insulation of the internal coil isprovided by a flexible silicone molding (not shown). In use, internalreceiver unit 212 may be positioned in a recess of the temporal boneadjacent to outer ear 201 of the recipient.

As shown in FIG. 1A, multimodal system 200 is further configured tointeroperate with a user interface 280 and an external processor 282such as a personal computer, workstation, or the like, implementing, forexample, a hearing implant fitting system. Although a cable 284 is shownin FIG. 1A between implant 200 and interface 280, a wireless RFcommunication may also be used along with remote 286.

While FIG. 1A shows a multimodal implant in the ipsilateral ear, inother embodiments of the present invention the multimodal implant mayprovide stimulation to both ears. For example, a signal processor mayprovide electrical stimulation to one ear and provide acousticalstimulation in the other ear.

Cochlea 232 is tonotopically mapped with each region of the cochleabeing responsive to acoustic and/or stimulus signals in a particularfrequency range. To accommodate this property of cochlea 232, thecochlear implant system includes an array of electrodes each constructedand arranged to deliver suitable stimulating signals to particularregions of the cochlea, each representing a different frequencycomponent of a received audio signal 203. Signals generated bystimulator unit 226 are applied by electrodes of electrode array tocochlea 232, thereby stimulating the auditory nerve.

Typically, the electrode array of the cochlear implant includes aplurality of independent electrodes each of which can be independentlystimulated. Low frequency sounds stimulate the basilar membrane mostsignificantly at its apex, while higher frequencies more stronglystimulate the basilar membrane's base. Thus, electrodes of electrodearray are located near the base of cochlea 232 are used to simulate highfrequency sounds while electrodes closer to the apex are used tosimulate lower frequency sounds. In some embodiments, only certainelectrodes corresponding to certain frequency ranges are stimulated(e.g., with respect to a recipient who suffers from higher frequencyhearing loss, the electrodes at the basilar membrane's base arestimulated, while those near the apex are not activated, and instead,the frequency allocation of the sound signal 203 is allocated to theacoustic hearing aid).

In at least some situations, the level and type of hearing loss of aperson is often almost exclusively assessed utilizing pure toneaudiometry which is presented in the form of an audiogram. By way ofexample only and not by way of limitation, such an audiogram can beutilized to determine which electrodes should be activated forstimulation and which frequency ranges should be reserved for acousticstimulation by the acoustic hearing aid.

Audiograms are often utilized as the first step in fitting a hearingaid, with some subjective changes afterwards. Such pure tone audiometryto provide a person's audiogram typically consumes the time of a trainedaudiologist, and often requires at least one fitting session, and somespecific audiometric equipment. For these reasons, these tests are noteasily able to be carried out without a trained professional in anaudiology clinic. In at least some exemplary embodiments according tothe teaching detailed herein, by analyzing the speech of a person, oneor more of the aforementioned “requirements” can be done away with. Inthis regard, people who have moderate to severe congenital deafness orearly childhood deafness have, in at least some scenarios, significantchanges to their speech production compared to their normal hearingpeers. More subtle changes in speech production are reported inpost-lingually deafened adults. Changes in speech production range fromstressed and unstressed pitch variability, midpoints of voicelessfricatives, and plosive spectral slope. In at least some exemplaryscenarios utilizing the teachings detailed herein, hearing loss has anumber of known and measurable effects on speech production.

At least some exemplary embodiments according to the teachings detailedherein utilize advanced learning signal processing techniques, which areable to be trained to detect higher order, and non-linear, statisticalproperties of signals. An exemplary signal processing technique is theso called deep neural network (DNN). At least some exemplary embodimentsutilize a DNN (or any other advanced learning signal processingtechnique) to analyze a person's speech to predict the likelihood thatthe person is suffering from hearing loss or from a change in his or herability to hear. At least some exemplary embodiments entail trainingsignal processing algorithms to detect subtle and/or not-so-subtlechanges, and provide an estimate of the hearing loss of the recipientand specific information thereabouts. That is, some exemplary methodsutilize learning algorithms such as DNNs or any other algorithm that canhave utilitarian value where that would otherwise enable the teachingsdetailed herein to analyze a person's speech to predict hearing healthoutcomes such as their hearing loss, and/or appropriate fittingparameters and technologies for hearing devices.

A “neural network” is a specific type of machine learning system. Anydisclosure herein of the species “neural network” constitutes adisclosure of the genus of a “machine learning system.” Whileembodiments herein focus on the species of a neural network, it is notedthat other embodiments can utilize other species of machine learningsystems accordingly, any disclosure herein of a neural networkconstitutes a disclosure of any other species of machine learning systemthat can enable the teachings detailed herein and variations thereof. Tobe clear, at least some embodiments according to the teachings detailedherein are embodiments that have the ability to learn without beingexplicitly programmed. Accordingly, with respect to some embodiments,any disclosure herein of a device, system constitutes a disclosure of adevice and/or system that has the ability to learn without beingexplicitly programmed, and any disclosure of a method constitutesactions that results in learning without being explicitly programmed forsuch.

Some of the specifics of the DNN utilized in some embodiments will bedescribed below, including some exemplary processes to train such DNN.First, however, some of the exemplary methods of utilizing such a DNN(or any other algorithm that can have utilitarian value) will bedescribed.

FIG. 2 depicts an exemplary flowchart for an exemplary method, method200, of utilizing a code of and/or from a machine learning algorithm,such as a DNN, according to an exemplary embodiment. Method 200 includesmethod action 210, which includes obtaining data based on speech of aperson. In an exemplary embodiment, method action 210 is executed suchthat a user provides a speech sample as input. The speech could be anyspeech, such as reading from a book or newspaper, talking on a phone, orgeneral dialogue. Any type of speech that can enable the teachingsdetailed herein can be utilized in at least some exemplary embodiments.In at least some exemplary embodiments, the speech sample can be alinguistically crafted piece of pose covering a wide linguistic rangesuch as “the rainbow passage” or “comma gets a cure,” by way of exampleonly and not by way of limitation. In at least some exemplaryembodiments, the speech sample can be recorded for processing by the DNN(or the code produced/from by the DNN).

That said, in an alternative embodiment, the action of obtaining databased on the speech of a person includes obtaining the data from anentity that obtained and/or analyzed the speech sample. That is, in anexemplary embodiment, to execute method action 210, the actor need notnecessarily be the person who directly obtains the speech sample.

It is also noted that in at least some exemplary embodiments, methodaction 210 can be executed such that the person who is the subject ofthe method action is that a remote location from the entity obtainingthe data based on the speech of the person. By way of example only andnot by way of limitation, in an exemplary embodiment, the personspeaking to a telephone, as noted above, and the telephone can transmitthe person's speech to a remote facility, anywhere in the world in someembodiments, where the person's speech, which is representative of thespeech of the person, is obtained upon receipt of the signal at leastbased on the signal generated by the person's telephone.

Method 200 further includes method action 220, which includes analyzingthe obtained data based on speech utilizing a code of and/or from amachine learning algorithm to develop data regarding hearing loss of theperson. Again, in an exemplary embodiment, the machine learningalgorithm can be a DNN, and the code can correspond to a trained DNNand/or can be a code from the DNN (more on this below).

The developed data regarding the hearing loss could be a measure of theperson's hearing health. This could be an estimate of the person'shearing loss in percentage or in dB attenuation or dB hearing loss. Anyunit of measure for any indicia that can have utilitarian value can beutilized in at least some exemplary embodiments. In an exemplaryembodiment, the developed data can correspond to an estimate of hearingloss at 500 Hz, 1 kHz, 2 kHz, and 4 kHz which can be utilized to createan audiogram, for example. In some exemplary embodiments, the developeddata can correspond to the developed audiogram.

FIG. 3 depicts an exemplary conceptual schematic of an example of method200, where speech is the input into a system that utilizes a trained DNNor some other trained learning algorithm (or the results thereof—thecode of a machine learning algorithm as used herein corresponds to atrained learning algorithm as used in operational mode after traininghas ceased and code from a machine learning algorithm corresponds to acode that is developed as a result of training of the algorithm—again,this will be described in greater detail below), and the output is anaudiogram.

It is noted that in at least some exemplary embodiments, any vocal inputcan be utilized with the system of FIG. 3. In at least some exemplaryembodiments, the whole vocal range is analyzed to develop the resultingproduction. That said, in some alternate embodiments, only portions ofthe vocal range are analyzed to develop the resulting production.

It is noted that in at least some exemplary embodiments, it is not thespeech signal that is provided directly to the learning algorithm.Instead, one or more feature extractions of the speech sample arecalculated and provided as inputs to the learning algorithm. Suchfeatures can, in some embodiments, take in the whole speech sample, andprovide a measure, or a set of measures from the speech sample. Forexample, overall loudness of the speech sample, the RMS, can be measurescorresponding to a single feature. Alternatively, and/or in addition tothis, loudness for one or more or any number of frequency bands can bedetermined, such as by calculating the power spectral density of thespeech for, for example, 20 frequencies, and these 20 frequency specificpowers are then provided to the algorithm. Alternatively, and/or inaddition to this, other features can be extracted, such as otherloudness measures, voicing measures such as pitch, jitter, pitch slope,pitch variability, etc., and/or articulatory features such as cepstrum,line spectral frequencies, and other analysis of the frequency spectrum.

Any data that is based on speech of a person can be utilized in at leastsome exemplary embodiments that can enable the teachings detailed hereincan be utilized as input into the signal analysis algorithm, and thusany data that can enable the teachings detailed herein that is based onspeech of a person can correspond to the data obtained in method action210.

FIG. 4 depicts another exemplary algorithm for another exemplary method,method 400, according to an exemplary embodiment. Method 400 includesmethod action 410, which includes executing method action numeral 210.Method 400 further includes method action 420, which includes obtainingbiographical data of the person. By way of example only, suchbiographical data can be non-speech or non-hearing related data, such asage, gender, native language, intelligence, or some socioeconomicmeasure. Also, such biographical data can be the time since the personhas lost his or her hearing or at least has had hearing problems, thetime that the person had hearing prior to losing hearing (or starting tolose hearing), the number of years that the person was lingual prior tolosing hearing, whether the person is lingual, etc. Any biographicaldata that can enable the teachings detailed herein and/or variationsthereof can be utilized in some embodiments.

Method 400 also includes method action 430, which has parallels in someembodiments to method action 220, except that in addition to utilizingthe obtained data based on speech, the method action 430 also utilizesthe obtained biographical data in the analysis by the code to developthe data regarding the person's hearing loss.

To be clear, the data based on speech can be both “raw” speech andfeatures extracted from that raw speech, or past speech or futurespeech. Indeed, in an exemplary embodiment, a speech therapist or thelike or a trained hearing specialist trained to evaluate speech and makeassumptions or estimates as to features of hearing loss can evaluatespeech of the person subjectively and/or objectively and provides suchdata into the DNN. Also, the DNN can receive the raw speech. The DNN canuse this data in combination to make the prediction as to the hearingloss of the recipient.

FIG. 5 presents a conceptual schematic of a DNN system depicting inputstherein and the output (prediction of hearing loss). As can be seen, inthis exemplary embodiment, information such as age, data relating to theonset of deafness (how long ago, how long since birth, type, suddenness,etc.), the gender of the recipient, and the raw speech of the recipientis inputted into the DNN. Also, speech is “pre-processed,” whether thatbe by a machine and/or by a human, and whether the speech that ispre-processed is the speech input into the DNN or whether that isdifferent speech than the speech input into the DNN.

In an exemplary embodiment, the more independent information containingspecific characteristics of the person provided to the learning model,the more accurate the prediction. To be clear, FIG. 5 depicts a systemwith a range of different inputs. Only one feature extraction input isshown (hearing loss prediction). However, in some alternate embodiments,there can be other feature extractions.

Note further that in some embodiments, there is no “raw speech” inputinto the DNN. Instead, it is all pre-processed data. Any data that canenable the DNN or other machine learning algorithm to operate can beutilized in at least some exemplary embodiments.

Note that the embodiments above are utilized to predict hearing healthmeasures. A range of hearing health or hearing benefit outputs can be,by way of example only and not by way of limitation:

-   -   An audiogram (few or many frequency measures)    -   A general level of deafness (a single overall level such as Pure        Tone Average)    -   A measure of the benefits that could be provided by a hearing        aid    -   A measure of the benefit that could be provided by a cochlear        implant or other device    -   An assessment of the conductive and neural hearing loss        components    -   A prediction of the etiology of hearing loss    -   A prediction of the age or length of profound hearing loss    -   A measure of the bilateral-ness of hearing loss

As noted above, method 200 and method 400 utilize a code from a machinelearning algorithm and/or a code of a machine learning algorithm. Inthis regard, the code can correspond to a trained neural network (thelatter). That is, as will be detailed below, a neural network can be“fed” statistically significant amounts of data corresponding to theinput of a system and the output of the system (linked to the input),and trained, such that the system can be used with only input, todevelop output (after the system is trained). This neural network usedto accomplish this later task is a “trained neural network.” That said,in an alternate embodiment, the trained neural network can be utilizedto provide (or extract therefrom) an algorithm that can be utilizedseparately from the trainable neural network. FIG. 6 depicts by way ofconceptual schematic exemplary “paths” of obtaining code utilized inmethods 200 and 400. With respect to the first path, the machinelearning algorithm 600 starts off untrained, and then the machinelearning algorithm is trained and “graduates” by symbolically crossingthe line 699, or matures into a usable code 600′—code of trained machinelearning algorithm. With respect to the second path, the code 610—thecode from a trained machine learning algorithm—is the “offspring” of thetrained machine learning algorithm 600′ (or some variant thereof, orpredecessor thereof), which could be considered a mutant offspring or aclone thereof. That is, with respect to the second path, in at leastsome exemplary embodiments, the features of the machine learningalgorithm that enabled the machine learning algorithm to learn may notbe utilized in the practice of method 200 or 400, and thus are notpresent in version 610. Instead, only the resulting product of thelearning is used.

In an exemplary embodiment, the code from and/or of the machine learningalgorithm utilizes non-heruistic processing to develop the dataregarding hearing loss. In this regard, the system that is utilized toexecute method 200 and/or 400 takes a speech signal in specifics ortakes data in general relating to speech, and extracts fundamentalsignal(s) there from, and uses this to predict hearing loss. In at leastsome exemplary embodiments, the prediction beyond a general “hearingloss” number. By way of example only and not by way of limitation, thesystem utilizes algorithms beyond a first-order linear algorithm, and“looks” at more than a single extracted feature. Instead, the algorithm“looks” to a plurality of features. Moreover, the algorithm utilizes ahigher order nonlinear statistical model, which self learns whatfeature(s) in the input is important to investigate. As noted above, inan exemplary embodiment, a DNN is utilized to achieve such. Indeed, inan exemplary embodiment, as a basis for implementing the teachingsdetailed herein, there is an underlying assumption that the features ofspeech and/or the other input into the system that enable the productionof hearing loss to be made are too complex to otherwise specified, andthe DNN is utilized in a manner without knowledge as to what exactly onwhich the algorithm is basing its prediction/at which the algorithm islooking to develop its prediction. Still further, in an exemplaryembodiment, the output is a prediction of an audiogram, as opposed togeneral hearing loss data.

In at least some exemplary embodiments, the DNN is the resulting codeused to make the prediction. In the training phase there are manytraining operations algorithms which are used, which are removed oncethe DNN is trained.

Note also, in at least some exemplary embodiments, the data developed inmethods 200 and 400 regarding hearing loss of the person is developedwithout identified speech feature correlation to the hearing loss.

More generally, according to the teachings detailed herein, genericfeatures are utilized, which features are associated with people beingdeaf or otherwise hard of hearing. No specific feature is utilized, orat least with respect to executing method 200 and/or 400 (or any of theother methods detailed herein), there is no specific feature of thespeech and/or the biographic data that is looked at. Thus, anon-heuristic processing method is utilized. To be clear, in at leastsome embodiments, the specific features utilized to execute method 200and/or 400 are not known (in some instances, they are not otherwisedescribed), and one does not need to care as to what specific featuresare utilized. In at least some exemplary embodiments, a learning systemis utilized which arbitrarily picks features within the input into thesystem in an attempt to “learn” how to predict hearing loss, and oncethe system learns how to predict hearing loss, it performs accordingly.

To be clear, in at least some exemplary embodiments, the trainedalgorithm is such that one cannot analyze the trained algorithm with theresulting code there from to identify what signal features or otherwisewhat input features are utilized to predict the hearing loss. This is asopposed to prior art predictive models that utilize frequency, forexample, frequency features to predict hearing. In this regard, in thedevelopment of the system, the training of the algorithm, the system isallowed to find what is most important on its own based on statisticallysignificant data provided thereto. In some embodiments, it is neverknown what the system has identified as important at the time that thesystems training is complete. The system is permitted to work itself outto train itself and otherwise learn to predict hearing loss.

An exemplary scenario of training the system will now be detailed.

Any learning model that is available and can enable the teachingsdetailed herein can be utilized in at least some exemplary embodiments.As noted above, an exemplary model that can be utilized with voiceanalysis and other audio tasks is the Deep Neural Network (DNN). Again,other types of learning models can be utilized, but the followingteachings will be focused on a DNN.

According to an exemplary embodiment of developing a learning model, alearning model type is selected and structured, and the features andother inputs (biographic, speech input, etc.) are decided upon and thenthe system is trained. It needs to be trained. In exemplary embodimentsof training the system, a utilitarian amount of real data is compiledand provided to the system. In an exemplary embodiment, the real datacomprises both sample speech (here, the single input, but in otherembodiments, additional input, such as the biographic input, can beutilized) and the data one is trying to protect (here, the person'saudiogram, for instance). Both the speech input and measure output(e.g., audiogram) are presented to the learning system (for one subjectat a time). The learning system then changes its internal workings andcalculations to make its own estimation closer to the actual person'shearing outcome. This internal updating of the model during the trainingphase can improve (and should improve) the system's ability to correctlypredict the groups output. Subsequent individual subject's inputs andoutputs are presented to the system to further refine the model. Withtraining according to such a regime, the model's predictive accuracy isimproved. In at least some exemplary embodiments, the larger and broaderthe training set, the more accurate the model becomes. An exemplaryspecific scenario is described below.

Thus, in this exemplary embodiment of the predictive model, both theinput and the output is collected and provided to the model to train themodel. In at least this exemplary embodiment, (i) the person's speechand (ii) a hearing health measure such as that person's audiogram areprovided as input for the training. This is repeated for a statisticallysignificant number of persons (e.g., 300 as used in the exemplaryscenario below). In at least some exemplary embodiments, as will bedescribed below, the input can also include or instead be the person'sspeech and that person's hearing aid fitting prescription/fit hearingaid settings (after 6 months of hearing aid acclimatization and finetuning) for fitting output models (as opposed to hearing outputmodels—again, an exemplary method of such will be described in greaterdetail below where the output is data for fitting a hearing prosthesisalternatively and/or in addition to the output being a prediction of therecipients ability to hear). In this last example, because it is notpredicting a hard measure, but a more complex selection of fittingparameters and technologies, the learning model output can actually bemore utilitarian on average than people's individual fittings previouslyreceived from hearing professionals. In an exemplary embodiment this canbe because the system provides an “average” output of a range ofprofessional's subjective fittings.

Still, focus for now will be on the scenario where the output of thetrained learning system is a prediction of the subject's audiogram fromthat person's speech production. With respect to this scenario, the datacollection for training and testing of the system can be as follows (byway of example only and not by way of limitation).

A subject (the subject used to train the system—again, this is trainingthe system, not the use of a trained system as would be used to executemethod 200 or 400) is instructed to speak for a statistically relevantamount of time (30 seconds, 1 minute, 90 seconds, 2 minutes, 3 minutes,4 minutes, 5 minutes, etc., or any value or range of values therebetweenin about one second increments, or more or less, reading from a book orfrom some form of prepared text. In an exemplary embodiment, the speechis recorded or otherwise captured by the sound capture device. In anexemplary embodiment, in addition to this, one or more demographicinformation data points are collected through a survey. All of this islinked to the subject. The subject is then tested with pure toneaudiometry to obtain the subjects audiogram. This provides the inputs(speech and biographical data) and the output (audiogram).

In at least some exemplary embodiments, a statistically relevant numberof subjects is utilized, and these subjects are representative of allthe subjects that are intended to be the subject of method 200 and/or400. In some embodiments, the data range of the subjects cover thefollowing characteristics:

-   -   The data represents a general population (levels of hearing        loss, age, gender, language, etc.)    -   The data contains members of each class (has multiple people in        each segment)    -   The data contains a wide range of variance (noise effect via the        utilization of multiple people in segment)

In some embodiments, with reference to the above, because the learningmodel of the system utilized in some embodiments is non-linear, and useshigh order statistics, it is not directly known what feature(s) of theinput are being used to make the prediction. Thus, in some embodiments,it is not possible to point to what in the input is the predictivefeature. This differentiates from some prior methods of predictinghearing loss based on speech, where one or more specific features or tolook towards in an effort to predict hearing loss. However, such canhave utilitarian value with respect to enabling the investigation anduse of a relatively large range of very complex features which are notable to be identified or even described.

In the case of a DNN, the size of the training can depend on the numberof neurons in the input layer, hidden layer(s), and output layer. Forinstance, a system with three layers (input, hidden, and output) couldhave by way of example and not by limitation the followingcharacteristics:

Input with a Total of 30 Neurons:

12 loudness measures across the frequency spectrum

-   -   12 spectral channels of modulation range    -   3 channels of format estimates (F0, F1, F2)    -   3 biographical inputs (length of hearing loss, age, gender)        Hidden Layer with a Total of 12 Neurons:    -   The input layer neurons would have connections from each input        and output neuron, as well as loop back and inter hidden layer        connections.        Output Layer with a Total of 5 Neurons:    -   A standard audiogram consisting of five spectral hearing loss        levels at 500, 1000, 2000, 4000 and 8000 Hz.

Given this described system, a reasonable minimum number of trainingsubject sets of data, spanning the subject spread for a simple feedforward DNN model could be:

Input*Hidden+Hidden*Output=30*12+12*5=420 subjects.

The example of the DNN model above can be trained with 420 people, butto test the accuracy of the developed model, additional data not used intraining is used. In this case the data can also span the subject group.A number around 100 additional subjects could be suitable as long asthere are multiple people in each segment. This data could be processedby the model, and the variance in the predicted outcome could becompared to actual audiogram of the verification subjects, and could bemeasured and used as the error (or prediction accuracy) of the model.After training and testing, the model would be well understood tooperate in predicting people's outcomes by only receiving their speechsample.

It is noted that in at least some exemplary embodiments, larger testsubject populations and/or smaller test subject populations can beutilized. In an exemplary embodiment, the test subject population can be100, 125, 150, 175, 200, 250, 300, 350, 400, 450, 500, 550, 600, 650,700, 750, 800, 850, or more test subjects. Any number of test subjectsthat can enable the teachings detailed herein can be utilized in atleast some exemplary embodiments.

There are many packages now available to perform the process of trainingthe model. Simplistically, the input measures are provided to the model.Then the outcome is estimated. This is compared to the subject's actualoutcome, and an error value is calculated. Then the reverse process isperformed using the actual subject's outcome and their scaled estimationerror to propagate backwards through the model and adjust the weightsbetween neurons, and improving its accuracy (hopefully). Then a newsubject's data is applied to the updated mode, providing a (hopefully)improved estimate. This is simplistic, as there are a number ofparameters apart from the weight between neurons which can be changed,but generally shows the typical error estimation and weight changingmethods for tuning models according to an exemplary embodiment.

FIG. 7 presents a conceptual schematic of a model of an exemplarycomplete system. A speech segment is processed to extract a number offeatures. These features as well as biographical information areprovided as inputs to the learning model. The trained model then processthese inputs to provide four outputs in this case (other outputs canexist). The outputs in this case are four pure tone thresholds at 500Hz, 1 kHz, 2 kHz, and 4 kHz. These four outputs are then used to createan accurate prediction of the person's audiogram.

In view of the above, FIG. 8 presents an exemplary flowchart for anexemplary method, method 800, of developing a predictive algorithm forhearing loss. In this exemplary embodiment, method 800 includes methodaction 810, which includes obtaining hearing data and speech data for astatistically significant number of individuals. This corresponds to,for example, the data of the 420 subjects detailed above by way ofexample only. Method 800 further includes method action 820, whichincludes analyzing the obtained hearing data and speech data using aneural network to develop a predictive algorithm for hearing loss basedon the results of the analysis, wherein the predictive algorithmpredicts hearing loss based on input indicative of speech of a hearingimpaired person who is not one of the individuals. Method action 820 isexecuted utilizing, for example, the DNN detailed above, although inother embodiments, any other type of machine learning algorithm can beutilized in at least some exemplary embodiments. Consistent with theteachings detailed above, in an exemplary embodiment, the predictivealgorithm is not focused on a specific speech feature. Instead, itutilizes a plurality of features that are unknown or otherwise generallyrepresent a complex arrangement (if such can even be considered featuresin the traditional sense).

Still further, in some exemplary embodiments of method 800, againconsistent with the teachings detailed above, the action of analyzingthe obtained hearing data machine-trains a system that results in thedeveloped predictive algorithm. Also, still with respect to method 800,in an exemplary embodiment, of the obtained hearing data and speechdata, a portion thereof is used, in the neural network, for training anda portion thereof is used, in the neural network, for verification.Also, in this exemplary embodiment, the neural network develops thepredictive algorithm by or transparently (sometimes, invisibly)identifying important features from the obtained hearing data and speechdata. Also consistent with the teachings above, the predictive algorithmutilizes an unknowable number of features present in the speech data topredict hearing loss.

With respect to the unknown/transparent features, these can be featuresthat are not related to a simple speech description. For instance,modulation is a term audiologists use to describe one aspect of speech.It may be that modulation is correlated with hearing loss. Whenunknowable and invisible/transparent are used to describe evaluatinghearing loss in this regard, it is meant that that hearing loss is nottypically (if at all) related to a feature, like modulation, which isknown. It is not unknowable in technical terms, since the DNN is adescription of the feature. This point is that this is not a simpleaudiological description of speech such as modulation, pitch, loudness,etc., or even a simple combination of these audiological descriptions,but the DNN will be looking at something which cannot be describedsimply by these terms, or a simple correlation or these terms.

FIG. 9A presents an exemplary algorithm for an exemplary method, method900, according to an exemplary embodiment. Method 900 includes methodaction 910, which includes executing method action 810. Method 900further includes method action 920, which includes obtainingbiographical data for the individuals of the statistically significantnumber of individuals of method action 910 (method action 810). Such canbe executed in accordance with the teachings detailed above. Method 900further includes method action 930, which generally parallels methodaction 820 detailed above, but includes, analyzing the obtained hearingdata and speech data and the obtained biographical data using a neuralnetwork to develop a predictive algorithm for hearing loss based on theresults of the analysis, wherein the predictive algorithm predictshearing loss based on input indicative of speech and biographical dataof a hearing impaired person who is not one of the individuals.

FIG. 9B depicts another exemplary method, method 940, according to anexemplary embodiment. The underlying idea behind method 940 is that theteachings utilized herein with respect to utilizing and train systems topredict hearing loss or the like can also be utilized to obtain datafrom a given subject to further train the DNN. In this regard,audiograms or the like can be developed or otherwise obtained in atraditional manner for a given subject of method 200 or 400. Thisaudiogram or other data would not be utilized to predict the hearingloss per se, but instead would be utilized in a manner consistent withthe training information detailed above (whether as a test subject fortraining or for model verification). Thus, in an exemplary embodiment,method 940 includes, in method action 950, executing method 800 and/ormethod 900 and then subsequently, in method action 960, executing method200 and/or 400. In this regard, the code developed in method action 950is utilized to execute method action 960. After method action 960 isexecuted, method action numeral 970 is executed, which includesobtaining data from the subject of method action 960, this datacorresponding to the data of method actions 210 and/or 420. Utilizingthis data, method 940 proceeds back to method action 950 and utilizesthe data obtained in method action 970 to execute (re-execute) method800 and/or method 900. It is noted that method action numeral 960 caninstead correspond to obtaining data from the results of the executionof method 200 and/or 400. That is, instead of executing method 200and/or 400, the actor who executes method 940 does not execute thosemethod actions, but instead obtains the data from someone else, directlyor indirectly, who is executed those method actions. FIG. 9C depictssuch an exemplary alternative method of method 940, method 945, whichincludes method action 950 which corresponds to method action 950detailed above. Method action numeral 980 corresponds to the action ofobtaining data relating to a subject of the execution of method 200and/or 400, where the code utilized to execute method 200 and/or 400 isthe code that results from the method action 950. In an exemplaryembodiment, the data corresponds to the data of method actions 210and/or 420 plus additional data, such as by way of example only and notby way of limitation, in audiogram obtained in a traditional manner.Method 945 and then goes back to method action 950, which entailsre-executing method 800 and/or method 900, but utilizing the dataobtained in method action 980 to further train and/or validate the DNN,etc.

Briefly, it is noted that the neural networks or other machine learningalgorithms utilized herein do not utilize correlation, or, in someembodiments, do not utilize simple correlation, but instead developrelationships. In this regard, the learning model is based on utilizingunderlying relationships which may not be apparent or otherwise evenidentifiable in the greater scheme of things. In an exemplaryembodiment, MatLAB, Buildo, etc., are utilized to develop the neuralnetwork. In at least some of the exemplary embodiments detailed herein,the resulting train system is one that is not focused on a specificspeech feature, but instead is based on overall relationships present inthe underlying statistically significant samples provided to the systemduring the learning process. The system itself works out therelationships, and there is no known correlation based on the featuresassociated with the relationships worked out by the system.

The end result is a code which is agnostic to speech features. That is,the code of the trained neural network and/or the code from the trainedneural network is such that one cannot identify what speech features areutilized by the code to develop the production (the output of thesystem). The resulting arrangement is a complex arrangement of anunknown number of features of speech that are utilized to predict thehearing loss of the subject. The code is written in the language of aneural network, and would be understood by one of ordinary skill in theart to be such, as differentiated from a code that utilized specific andknown features. That is, in an exemplary embodiment, the code looks likea neural network.

Consistent with common neural networks, there are hidden layers, and thefeatures of the hidden layer are utilized in the process to predict thehearing impediments of the subject.

The developed data of method 200 and 400, or, more accurately,variations thereof, could also be a less tangible measure, such as byway of example only and not by way of limitation, a factor or a range offactors as could be useful to determine a suitable hearing aid fitting.In this way, for example, the teachings detailed herein could bypass theintermediate process of using an audiogram to estimate a hearing aidfitting, and then have additional audiological tuning steps, and providea final fitting estimate for the person (recipient). Some additionaldetails of this are now described.

FIG. 10 depicts an exemplary flowchart for an exemplary method, method1000, which includes method action 1010, which includes obtaining databased on speech of a person. In this regard, method action 1010 cancorrespond to method action 210 and/or 410 detailed above. Indeed, in anexemplary embodiment, method 1000 can be an offshoot of method 200and/or 400 detailed above. That is, an exemplary method can start offwith method 210, and can then branch off by executing the remainingactions of method 200 and 400 on the one hand, and the following actionsof method 1000 on the other hand. With respect to the following actions,in an exemplary embodiment, method 1000 further includes method action1020, which includes developing a prescription and/or a fitting regimefor a hearing prosthesis based on the obtained data obtained in methodaction 1010. As noted above, in an exemplary embodiment, method 1000completely skips the results of method 200 and 400. Instead, the resultis output corresponding to how the hearing prosthesis should be fittedto the recipient. Also, consistent with the teachings detailed abovethat utilize a neural network, in an exemplary embodiment, theprescription can be developed based on relationships as opposed tocorrelations between speech and hearing loss. That said, it is notedthat in at least some exemplary embodiments, method 1000 is executedwithout utilizing a neural network or otherwise some form of machinelearning algorithm or code based thereon. Still, in at least someexemplary embodiments, method 1000 is executed by utilizing a codewritten in the language of a neural network to develop the prescriptionand/or fitting regime.

Consistent with utilization of the method 1000 for fitting aconventional hearing aid, in an exemplary embodiment, the prescriptionand/or fitting regime is a gain model. In this regard, an exemplaryembodiment, this gain model can be applied to the hearing prostheses todetermine what gain should be applied at least for certain frequencies.Note also that such can be applicable to other types of hearingprostheses, such as by way of example only and not by way of limitation,cochlear implants, middle ear implants, bone conduction devices etc. Anyprosthesis to which the teachings detailed herein can be applied can beutilized in at least some exemplary embodiments.

It is further noted that in at least some exemplary embodiments, theresults of method 1000 (or method 1100, as will be described below) areutilized via input via the user interface 280 and an external processor282 of FIG. 1A to fit the hearing prosthesis. In this regard, anexemplary embodiment includes utilizing the results of method 1000 (ormethod 1100), or any of the other applicable methods detailed herein, toprogram or fit the hearing prosthesis utilizing the arrangement of FIG.1A.

FIG. 11 represents an exemplary flowchart for an exemplary method,method 1100, which generally parallels method 1000 above, but which alsoutilizes non-speech and non-hearing related data. In this regard, method1100 includes method 1110, which entails executing method action 1010.Method 1100 also includes method action 1120, which includes obtainingnon-speech and non-hearing related data. In an exemplary embodiment,this can correspond to the biographic information detailed above. Anynon-speech and/or non-hearing related data that can enable the teachingsdetailed herein in general, and can enable method 1100 in particular,can be utilized in at least some exemplary embodiments. Method 1100further includes method action 1130 which includes developing aprescription and/or a fitting regime for the hearing prostheses based onthe obtained data for method action 1110 and from method action 1120. Inthis regard, method 1100 generally parallels method 400 detailed abovewhich utilizes the biographic data. It is noted that method 1000 andmethod 1100 can be executed so as to bypass the development of anaudiogram or otherwise the developments of data relating to the hearingdeficiencies of the subject. Instead, in an exemplary embodiment, theaction of developing a prescription and/or the fitting regime isexecuted directly from the obtained data obtained in method actions 1110and/or 1120. This is as opposed to executing such indirectly from theobtained data (e.g., via the use of an intervening audiogram to developthe prescription and/or fitting regime).

Thus, an exemplary output of an exemplary system (utilizing code writtenin the language of a machine learning algorithm or otherwise) can useany of the measures detailed herein to determine a hearing prosthesis'sfitting, and to determine such fitting directly. The fitting is thespecific parameters and technologies to be used by the hearingprosthesis during use for a particular recipient. For instance, a numberof measures could be used to determine the gain system settings as notedabove, but also some of the other signal processing technology settings.Also, suitable automation settings can be utilized in some exemplaryembodiments. In this way, a set of hearing aid parameters could bedetermined directly from a recipient's (or future recipient's) input.This can comprise two or more separate models, where the internalmeasures of hearing can be provided to a second fitting model. Or suchcan comprise a single learning model which has no internal knowledge ofthe measures typically used in predicting fitting parameters.

Thus, in contrast to some prior methods of fitting a hearing prosthesis,instead of obtaining an audiogram and, based on the audiogram,developing a fitting regime/prescription for a hearing prosthesis, theteachings detailed herein can skip the acquisition of an audiogram, oreven the development of specific data indicative of a person's hearingloss, and instead, a prescription/fitting regime can be directlydeveloped from the inputs. While the teachings detailed herein focus onsome embodiments of the utilization of a learning model, in otherembodiments, a learning model is not utilized. Still, in at least someexemplary embodiments, a learning model is utilized to predict a fittingregime or prescription based on the input in a manner analogous to orotherwise corresponding to that which was applied in developing thesystems to protect hearing loss. In this regard, in an exemplaryembodiment, any of the teachings detailed herein with respect todeveloping the system to protect hearing loss and/or the utilization ofsuch system to predict hearing loss also corresponds to teachings thatare applicable to the development of a system to predict a fittingregime and/or a prescription and/or the utilization of such a system toprotect the fitting regime and/or prescription.

With respect to some specifics of a predicted fitting regime and/orprescription, in an exemplary embodiment, the results of methods 1000and/or 1100 or the other methods detailed herein is a NAL prescriptionin whole or in part. In an exemplary embodiment, the prescription can befor the conventional hearing aid and/or for a cochlear implant, or anyother hearing prosthesis. In an exemplary embodiment, such as where thefitting regime is utilized for a cochlear implant, a noise reductionfitting regime and/or a gain fitting regime can be the result of methods1000 and 1100. Still further, the result can be a predicted gain modelfor the hearing prostheses. Note also that as will be described ingreater detail below, in an exemplary embodiment, the prescription iseliminated in its entirety. Instead, the result is a fitting regime fora hearing prosthesis that is directly applied to the hearing prosthesis(e.g., the fitting regime constitutes programming for the hearingprostheses that when applied to the hearing prostheses, the hearingprostheses configures itself to operate accordingly—this can be entirelytransparent to the recipient and/or the healthcare professional). Insuch embodiments that apply such, along with the embodiments that applysuch and also skip the development of the audiogram, streamlining of thefitting process can be achieved.

In an exemplary embodiment, the results of methods 1000 and/or 1100 area prescription and/or a fitting regime that enables the hearingprosthesis of U.S. Pat. No. 4,803,732 to Dillon (issued Feb. 7, 1989)and/or the hearing prosthesis of U.S. Pat. No. 5,278,912 to Waldhauer(issued Jan. 11, 1994), to be fitted or adjusted per the prescription.In this regard, in an exemplary embodiment, the resulting prescriptionand/or fitting regime includes data for adjusting one or more featuresof the respective hearing prostheses of these patent documents. Also,the results of method 1000 and/or 1100 can be a prescription and/or afitting regime that provides for sound compression/signal compressionfor the pertinent subject. By way of example only and not by way oflimitation, the results of method 1000 and/or 1100 can enable the inputcontrolled compression to be utilitarian or otherwise controlled for aspecific recipient. Also by way of example, the results of method 1000and/or 1100 can be that which enables or otherwise identifies theutilitarian fitting compression for a given hearing prostheses for thatrecipient.

To be clear, in some exemplary embodiments, the results of method 1000and/or 1100 and/or the results of methods 200 and 400 can be used todevelop hearing prosthesis data such as hearing prosthesis frequencyspecific gains, a hearing prosthesis gain system, the selection ofsuitable processing technologies or the level (strength) of a range ofhearing technologies (e.g., noise reduction, directional microphoneapplication, and the weighting of the given technology relative to oneanother (if used in combination) etc.) etc.

Also, in an exemplary embodiment, the unilateral and/or bilateralamplifications can be determined as a result of methods 1000 and/or1100. Of course, in some embodiments, the results can be a prescriptionof an amplification regime for a given hearing prosthesis.

In view of the above, in an exemplary embodiment, the results of method1000 and/or 1100 can be the settings of a given hearingprosthesis/programming of a given hearing prosthesis and/or data forsetting a given hearing prosthesis or programming a given hearingprosthesis with respect to a gain model, feedback management, frequencycompression, noise reduction, etc., for a given recipient or subjectbased on the data of method actions 1010 and/or 1120.

FIG. 12 presents an exemplary conceptual schematic according to such anexemplary embodiment utilized for fitting/prescription development. Morespecifically, FIG. 12 presents an exemplary learning algorithm which hastwo stages: the first stage is to predict the hearing health outcome andmeasure such as an audiogram. The second stage is to determine the mostsuitable hearing aid fitting/prescription.

Thus, in view of the above, utilizing speech input data, whether in rawformat or in a preprocessed format, or in combination, along withnon-speech and/or non-hearing related data, such as age, gender, etc.,the teachings utilize herein can be applied to predict or otherwisedevelop data relating to a hearing loss of a subject and/or to predictor otherwise develop data relating to fitting of a hearingprosthesis/the development of a prescription for a hearing prosthesis.As noted above, in an exemplary embodiment, separate methods can bepracticed where the output is respectively data relating to hearing lossand data relating to how to program or otherwise fit a given hearingprostheses, whereas in some exemplary embodiments, the methods can bepracticed in combination with the output is both the hearing loss andthe programming information. Still further, as noted above, in someembodiments, the hearing loss data can be skipped in its entirety, and,in some embodiments, the output can be entirely directed towardsprogramming information/fitting information.

The below is a chart presenting high-level concepts of the types ofpredictions that can be made utilizing the teachings detailed hereinand/or variations thereof, along with exemplary input and output forsuch predictions. It is also noted that the input and output alsocorresponds to the input when developing a predictive model for suchpredictions:

Description Input Output General hearing loss Speech (from PTA (averageof HL at prediction a paragraph) 0.5, 1, 2 and 4 kHz) Audiogramprediction Speech, biographical Audiogram information Fitting of gainsystem Speech, biographical Gains, knee points and (HA, or CI)information time constants Fitting of technology Speech, biographicalSettings for dir. mics, use (HA, or CI) information NR systemsPredicting Benefit in Speech, biographical Words correct in quiet speechunderstanding information or in noise Predicting Benefit in Speech,biographical SSQ, QoL survey quality of life measure information resultsConductive vs Speech, biographical Level of each, or sensorineurallevels information proportion of each Hearing loss etiology Speech,biographical Etiology of hearing loss information, biometric Determinethe length Speech, biographical Number of years of mild of hearing lossinformation and severe hearing loss Bilateral hearing Speech,biographical Difference in hearing information loss between ears

An exemplary data flow used to create a predictive configuration modelthat is usable in an automatic prosthesis configuration program and/orautomatic hearing impairment data identification will now be described.In this regard, in an exemplary embodiment, the teachings detailedherein can be utilized or otherwise can be modified to be utilized in asituation where a recipient is wearing a hearing aid (a conventionalhearing aid) and/or a cochlear implant (or other type of hearingprosthesis) so as to detect through the recipient speech if therecipient's hearing prostheses is not restoring his or her hearing to adesired/optimal level. By way of example, the recipient wears thehearing prostheses during testing, and the teachings detailed hereinand/or variations thereof are utilized to predict, from the recipientspeech, changes that could be implemented to the hearing prostheses soas to improve hearing. By way of example only and not by way oflimitation, a change to the map or a change to the processing regime,etc. It is further noted that in at least some exemplary embodiments,the data that is utilized to train the machine learning systems detailedherein can be developed utilizing a statistically significant populationthat has a hearing aid and/or a cochlear implant, etc.

Note that in some embodiments, the teachings detailed herein areimplemented in a three stage process. First, the teachings detailedherein are implemented with respect to people who do not have a hearingprosthesis. These teachings are utilized to determine the suitability orotherwise utilitarian value with respect to having a given recipientutilize a conventional hearing aid and/or with respect to fitting aconventional hearing aid there to. Such can also be the case withrespect to determining the suitability or otherwise utilitarian valuewith respect to having a given recipient utilize a bone conductiondevice, which device can be utilized with recipients who are hard ofhearing, but still hear (analogous to recipients who have suitabilityfor a conventional hearing aid but who do not have sufficient hearingimpairment to warrant a cochlear implant). Next, the teachings detailedherein are implemented with respect to people who have a conventionalhearing aids (and/or, in the exemplary scenario, bone conductiondevices), to determine the suitability or otherwise utilitarian valuewith respect to having a given recipient utilize a cochlear implant, orotherwise with respect to fitting such cochlear implant.

In this example, a predictive configuration model is based on datamining principles created from large sets of empirical data. In thisexample, input training data is used to build the predictiveconfiguration model and can include many instances of individual sets ofmatching data, which have been empirically obtained from a large numberof cochlear implant recipients, conventional hearing aid recipients,bone conduction device recipients, middle ear recipients, etc. Theseindividual sets of matching data can include objective physicalmeasurements, training configuration variables, and stimulationparameters.

More specifically in this example, objective physical measurements caninclude any one or more of the characteristics earlier identified.Similarly, the training configuration variables can include, but are notlimited to subjectively determined values. A machine learning algorithmthen undergoes a “training” process to determine one or more valuesrelated to fitting the hearing prosthesis, a prescription for a hearingprosthesis and/or related to hearing impairments of the recipient.

It is noted that in an exemplary embodiment, the input data for any ofthe methods detailed herein can be obtained via a dedicated speechtesting regime, which can include sitting in front of a personalcomputer or tablet or the like, or can include the subject speaking intoa telephone and/or inputting non-speech data/biographic data, into acomputer. Such can be executed in so called “point of care” sessions inthe clinical setting and/or in so-called “remote care” settings, at homeor remote from a central care provider. Thus, in an exemplaryembodiment, the raw speech upon which the various methods detailedherein are based can be captured at a clinic and/or remotely, using ahearing prosthesis or other devices (phone, dedicated microphone, etc.),which can capture the sound and convey it to the system that isundergoing training and/or the trained system/system using the code fromthe train system. Again, additional data can be collected at the pointwhere the raw speech is captured or subsequently which additional datacan be by way of example only and not by way of limitation, patientspecific information such as age, etiology, genetic information, andinformation about any hearing prosthesis being used.

The smart phone or other remote component can analyze the data streamthereto to extract data and/or can act as a medium to pass the datastream thereto to the cloud for cloud computing. That is, the smartphone or other remote device passes the collected information from theremote device to the cloud.

The system includes a link from the cloud to a clinic to pass theinformation uploaded to the cloud to the clinic, where the informationcan be analyzed. Another exemplary system includes a smart device, suchas a smart phone or tablet, etc., that includes a sound capture device,that is running a purpose built application to implement some of theteachings detailed herein. In this exemplary system, the hearingprosthesis is bypassed or otherwise not utilized. Indeed, in thisexemplary system, a hearing prosthesis may not be utilized at all. Thatis, in an exemplary embodiment, the person that is speaking is not arecipient of the hearing prosthesis, but instead a subject who could behard of hearing. In an exemplary embodiment, the smart device can beconfigured to present a series of words for the recipient to repeat,which words are words that have been preselected for the purpose ofidentifying hearing attributes based on the speech of hearer/subject. Inthis regard, the hearer/subject vocalizes the visual signals that arepresented on the smart device (e.g., speaks the words displayed on thesmart device). The subject's speech is then picked up by the smartdevice's microphone. The smart device either uploads the speech to thecloud or otherwise analyzes the speech and uploads the analysis and/orraw speech or other data to the cloud. Alternatively, the smart devicecan analyze the speech autonomously according to the teachings detailedherein. In an exemplary embodiment, the cloud stores the results of thetests across all subjects, and data can be sent to the relevant clinicfor use in the methods detailed herein.

It is noted that there can be a wide variety of data (input data)collection techniques and/or acquisition techniques, whether the data beutilized for the methods detailed herein with respect to developingfitting data for a prosthesis or for ascertaining a hearing impairmentof a subject, or for developing the data to train the learningalgorithm's detailed herein. Such can entail the utilization of a smartphone, a dedicated microphone, a personal computer, a landline phone,etc. In this regard, in an exemplary embodiment, the data can beobtained from remote locations and analyzed at a different location. Itis also noted that a wide variety of data output utilization or transfertechniques can be utilized. By way of example only and not by way oflimitation, in an exemplary embodiment, some of the teachings detailedherein can be utilized to remotely fit a hearing prosthesis. FIG. 13presents a functional schematic of a system with which some of theteachings detailed herein and/or variations thereof can be implemented.In this regard, FIG. 13 is a schematic diagram illustrating oneexemplary arrangement in which a system 1206 can be used to execute oneor more or all of the method actions detailed herein in conjunction withthe use of a device 100, which can be a hearing prosthesis, or can be apersonal computer or a phone, etc. Herein, device 100 will often bereferred to as a hearing prosthesis, but note that that can be a proxyfor any of the devices that can enable the data acquisition according tothe teachings detailed herein. In general, in an exemplary embodiment,the system of FIG. 13 can be representative of both the system utilizedto develop the model (to train the model) and the system that resultsfrom the model.

System 1206 will be described, at least in part, in terms of interactionwith a recipient, although that term is used as a proxy for anypertinent subject to which the system is applicable (e.g., the testsubjects used to train the DNN, the subject utilized to validate thetrained DNN, the subjects to which methods 200, 400, 1000 and 1100 areapplicable, etc.). In an exemplary embodiment, system 1206 is arecipient controlled system while in other embodiments, it is a remotecontrolled system. In an exemplary embodiment, system 1206 cancorrespond to a remote device and/or system, which, as detailed above,can be a portable handheld device (e.g., a smart device, such as a smartphone), and/or can be a personal computer, etc.

In an exemplary embodiment, system 1206 can be a system havingadditional functionality according to the method actions detailedherein. In the embodiment illustrated in FIG. 13, the device 100 can beconnected to system 1206 to establish a data communication link 1208between the hearing prosthesis 100 (where hearing prosthesis 100 is aproxy for any device that can enable the teachings detailed herein, suchas a smartphone with a microphone, a dedicated microphone, a phone,etc.) and system 1206. System 1206 is thereafter bi-directionallycoupled by a data communication link 1208 with hearing prosthesis 100.Any communications link that will enable the teachings detailed hereinthat will communicably couple the implant and system can be utilized inat least some embodiments.

System 1206 can comprise a system controller 1212 as well as a userinterface 1214. Controller 1212 can be any type of device capable ofexecuting instructions such as, for example, a general or specialpurpose computer, a handheld computer (e.g., personal digital assistant(PDA)), digital electronic circuitry, integrated circuitry, speciallydesigned ASICs (application specific integrated circuits), firmware,software, and/or combinations thereof. As will be detailed below, in anexemplary embodiment, controller 1212 is a processor. Controller 1212can further comprise an interface for establishing the datacommunications link 1208 with the hearing prosthesis 100 (again, whichis a proxy for any device that can enable the methods herein—any devicewith a microphone and/or with an input suite that permits the input datafor the methods herein to be captured). In embodiments in whichcontroller 1212 comprises a computer, this interface may be, forexample, internal or external to the computer. For example, in anexemplary embodiment, controller 1206 and cochlear implant may eachcomprise a USB, FireWire, Bluetooth, Wi-Fi, or other communicationsinterface through which data communications link 1208 may beestablished. Controller 1212 can further comprise a storage device foruse in storing information. This storage device can be, for example,volatile or non-volatile storage, such as, for example, random accessmemory, solid state storage, magnetic storage, holographic storage, etc.

User interface 1214 can comprise a display 1222 and an input interface1224 (which, in the case of a touchscreen of the portable device, can bethe same). Display 1222 can be, for example, any type of display device,such as, for example, those commonly used with computer systems. In anexemplary embodiment, element 1222 corresponds to a device configured tovisually display a plurality of words to the recipient 1202 (whichincludes sentences), as detailed above.

Input interface 1224 can be any type of interface capable of receivinginformation from a recipient, such as, for example, a computer keyboard,mouse, voice-responsive software, touchscreen (e.g., integrated withdisplay 1222), microphone (e.g. optionally coupled with voicerecognition software or the like) retinal control, joystick, and anyother data entry or data presentation formats now or later developed. Itis noted that in an exemplary embodiment, display 1222 and inputinterface 1224 can be the same component, e.g., in the case of a touchscreen). In an exemplary embodiment, input interface 1224 is a deviceconfigured to receive input from the recipient indicative of a choice ofone or more of the plurality of words presented by display 1222.

It is noted that in at least some exemplary embodiments, the system 1206is configured to execute one or more or all of the method actionsdetailed herein, where the various sub-components of the system 1206 areutilized in their traditional manner relative to the given methodactions detailed herein.

In an exemplary embodiment, the system 1206, detailed above, can executeone or more of the actions detailed herein and/or variations thereofautomatically, at least those that do not require the actions of arecipient.

While the above embodiments have been described in terms of the portablehandheld device obtaining the data, either directly from the recipientor from the hearing prosthesis, and performing a given analysis, asnoted above, in at least some exemplary embodiments, the data can beobtained at a location remote from the recipient, and thus the device100. In such an exemplary embodiment, the system 1206 can thus alsoinclude the remote location (e.g., clinic).

In this vein, it is again noted that the schematic of FIG. 13 isfunctional. In some embodiments, a system 1206 is a self-containeddevice (e.g., a laptop computer, a smart phone, etc.) that is configuredto execute one or more or all of the method actions detailed hereinand/or variations thereof. In an alternative embodiment, system 1206 isa system having components located at various geographical locations. Byway of example only and not by way of limitation, user interface 1214can be located with the recipient (e.g., it can be the portable handhelddevice 240) and the system controller (e.g., processor) 1212 can belocated remote from the recipient. By way of example only and not by wayof limitation, the system controller 1212 can communicate with the userinterface 1214, and thus the portable handheld device 240, via theInternet and/or via cellular communication technology or the like.Indeed, in at least some embodiments, the system controller 1212 canalso communicate with the user interface 1214 via the Internet and/orvia cellular communication or the like. Again, in an exemplaryembodiment, the user interface 1214 can be a portable communicationsdevice, such as, by way of example only and not by way of limitation, acell phone and/or a so-called smart phone. Indeed, user interface 1214can be utilized as part of a laptop computer or the like. Anyarrangement that can enable system 1206 to be practiced and/or that canenable a system that can enable the teachings detailed herein and/orvariations thereof to be practiced can be utilized in at least someembodiments.

In view of the above, FIG. 14 depicts an exemplary functional schematic,where a device 2240, which will be detailed herein in this exemplaryembodiment as a portable hand-held device 2240, but is to be understoodas representative of any device that can enable the teachings detailedherein (e.g., remote dedicated hearing prosthesis control unit, personalcomputer, smartphone, landline phone, etc.) is in communication with ageographically remote device/facility 10000 via link 2230, which can bean internet link. The geographically remote device/facility 1000 canencompass controller 1212, and the remote device 240 can encompass theuser interface 1214. The geographic remote device/facility 10000 can bethe clinic. It is also noted that in the embodiment of FIG. 18, link2230 can represent communication between the portable handheld device2240 and the hearing prosthesis 100 and/or can represent communicationbetween the portable handheld device 2240 and the subject (bypassing thehearing prosthesis).

Accordingly, an exemplary embodiment entails executing some or all ofthe method actions detailed herein where the recipient of the hearingprosthesis or other subject, the hearing prosthesis 100 and/or theportable handheld device 2240 is located remotely (e.g., geographicallydistant) from where at least some of the method actions detailed hereinare executed.

In an exemplary embodiment, the portable handheld device 2240 isconfigured to execute one or more of the method actions detailed herein.In an exemplary embodiment, the portable handheld device 2240 isconfigured to communicate with the cloud as detailed above and/or withthe clinic as detailed above.

Reference herein is frequently made to the recipient of a hearingprosthesis. It is noted that in at least some exemplary embodiments, theteachings detailed herein can be applicable to a person who is not therecipient of a hearing prosthesis. Accordingly, for purposes ofshorthand, at least some exemplary embodiments include embodiments wherethe disclosures herein directed to a recipient correspond to adisclosure directed towards a person who is not a recipient but insteadis only hard of hearing or otherwise has a hearing ailment.

Any disclosure herein of the hearing prosthesis executing one or more ofthe method actions detailed herein are having a disclosed functionalityalso corresponds to a disclosure of a remote device and/or a personexecuting those method actions. That is, by way of example only and notby way of limitation, the actions of the hearing prosthesis can beperformed by another device, such as a smart phone, a personal computer,etc. Also, any disclosure of any remote device executing one or more themethod actions detailed herein or otherwise having a disclosedfunctionality also corresponds to a disclosure of a hearing prosthesishaving such functionality and/or being configured to execute such methodactions, along with a disclosure of a person executing such methodactions.

Any disclosure of any method action detailed herein corresponds to adisclosure of a device and/or a system for executing that method action.Any disclosure of any method of making an apparatus detailed hereincorresponds to a resulting apparatus made by that method. Anyfunctionality of any apparatus detailed herein corresponds to a methodhaving a method action associated with that functionality. Anydisclosure of any apparatus and/or system detailed herein corresponds toa method of utilizing that apparatus and/or system. Any feature of anyembodiment detailed herein can be combined with any other feature of anyother embodiment detailed herein providing that the art enables such,and it is not otherwise noted that such is not the case.

While various embodiments of the present invention have been describedabove, it should be understood that they have been presented by way ofexample only, and not limitation. It will be apparent to persons skilledin the relevant art that various changes in form and detail can be madetherein without departing from the scope of the invention.

What is claimed is:
 1. A method, comprising: obtaining hearing data andspeech data for a statistically significant number of individuals; andanalyzing the obtained hearing data and speech data using machinelearning to develop a predictive algorithm for hearing loss based on theresults of the analysis, wherein the predictive algorithm predictshearing loss based on input indicative of speech of a hearing impairedperson who is not one of the individuals.
 2. The method of claim 1,wherein: the predictive algorithm is not focused on a specific speechfeature.
 3. The method of claim 1, wherein: the action of analyzing theobtained hearing data machine-trains a system that results in thedeveloped predictive algorithm.
 4. The method of claim 1, wherein: ofthe obtained hearing data and speech data, a portion thereof is used, ina neural network, for training and a portion thereof is used, in theneural network, for verification.
 5. The method of claim 1, wherein: themachine learning develops the predictive algorithm by internallyidentifying important features from the obtained hearing data and speechdata.
 6. The method of claim 1, wherein: the predictive algorithmutilizes an complex arrangement number of features present in the speechdata to predict hearing loss.
 7. The method of claim 1, furthercomprising: obtaining biographical data for the individuals of thestatistically significant number of individuals; analyzing the obtainedhearing data and speech data and the obtained biographical data using aneural network to develop a predictive algorithm for hearing loss basedon the results of the analysis, wherein the predictive algorithmpredicts hearing loss based on input indicative of speech andbiographical data of a hearing impaired person who is not one of theindividuals.
 8. A method, comprising: obtaining data based on speech ofa person; and analyzing the obtained data based on speech using a codeof and/or from a machine learning algorithm to develop data regardinghearing loss of the person, wherein the machine learning algorithm is atrained system trained based on a statistically significant populationof hearing impaired persons.
 9. The method of claim 8, wherein: the codeutilizes non-heuristic processing to develop the data regarding hearingloss.
 10. The method of claim 8, wherein: the data is developed withoutidentified speech feature correlation to the hearing loss.
 11. Themethod of claim 8, wherein: the data regarding hearing loss of theperson is an audiogram.
 12. The method of claim 8, wherein: the code ofthe machine learning algorithm is a trained neural network.
 13. Themethod of claim 8, wherein: the code is agnostic to the speech features.14. The method of claim 8, further comprising: obtaining biographicaldata of the person; and analyzing the obtained data based on speech andthe obtained biographical data using the code from the machine learningalgorithm to develop the data regarding hearing loss of the person. 15.A method, comprising: obtaining data based on speech of a person; anddeveloping a prescription and/or a fitting regime for a hearingprosthesis based on the obtained data.
 16. The method of claim 15,wherein: the prescription is developed based on relationships as opposedto correlations between speech and hearing loss.
 17. The method of claim15, wherein: the prescription and/or fitting regime is a gain model. 18.The method of claim 15, further comprising: utilizing a code written inthe language of a neural network to develop the prescription and/orfitting regime.
 19. The method of claim 15, wherein the action ofdeveloping the prescription and/or the fitting regime is executeddirectly from the obtained data.
 20. The method of claim 15, furthercomprising: obtaining non-speech and non-hearing related data anddeveloping the prescription and/or fitting regime based on thenon-speech and non-hearing related data.